Motion analysis method

ABSTRACT

The present invention relates to an automated system of measuring and assessing an individual&#39;s improvement in motion (for example, a patient&#39;s recovery) through exercise performance. In an embodiment of the system of the present invention, two approaches for monitoring movement are introduced for quantifying an individual&#39;s performance. Both approaches consider the control population as their reference and consider the difference, or what is referred to in this application as the distance, between the individual&#39;s data and the control population as the measure of performance.

FIELD OF THE INVENTION

The present invention relates to an automated system of measuring andassessing physical mobility through exercise performance.

BACKGROUND

Mobility improvement for patients is one of the primary goals ofphysiotherapy rehabilitation. Providing the physiotherapist and thepatient with a quantified and objective measure of progress can bebeneficial for monitoring the patient's condition.

The application of machine learning techniques to human motion analysishas grown rapidly in recent years. Measurement and analysis ofphysiotherapy data has the potential to provide an objective andquantitative measure of a patient's progress and improvement in motionperformance over the course of his or her physiotherapy treatment.During a typical physiotherapy session, a physiotherapist instructs thepatient to perform a number of exercises, each with several repetitions.The set of exercises chosen and the number of repetitions may becustomized for each patient. The physiotherapist then evaluates thepatient's progress based on their performance. As the patient'scondition improves the patient's performance also improves.

In current clinical practice, the patient's condition is typicallyassessed using visual observation of the patient's motions,questionnaires, and goniometry. Questionnaires such as the CommunityBalance and Mobility Scale and the Falls Efficiency Scale are used toassign a score to the patient's motion quality. Goniometry is atechnique of measuring joint angles which isolates a single body jointin order to evaluate a subject's range of motion. Goniometry is notaccurate when the subject is moving e.g., during exercises andfunctional rehabilitation.

The current measurement and assessment techniques requirephysiotherapist effort and monitoring, and are not capable of measuringduring movement. Automation of patient observation would supportphysiotherapy practice through automated assessment and evaluation ofexercise performance.

An automated system could also provide the therapist with numericalmetrics to assess the patient's recovery process and potentially allowphysiotherapists to assess the effectiveness of various treatmentprotocols over a population of patients.

Patient data analysis for progress monitoring is a challenging taskbecause of the complexity of human motion. Human movement consists ofsynchronous recruitment of multiple degrees of freedom (DoF), makingsingle DoF comparisons (e.g., only comparing the range of motion in onejoint) incomplete and possibly unreliable. Furthermore there are manysources of variability in human motion data.

For a single individual performing several repetitions of the samemovement, there is both spatial and temporal variability. Thisvariability is due to the nature of human movement, each repetition ofthe same exercise will be different, due to the stochastic nature ofmuscle recruitment. Interpersonal variability is due to differences inthe physical characteristics of different individuals, such asdifferences in height, weight, fitness level, etc. The measurementsystem, such as sensor noise and a gyroscope drift cause variability inthe patient motion data. Initial pose and sensor positioning couldintroduce variability into patient data. Recovery and progress causechanges in the patient motion data over the course of treatment. Aswell, fatigue and tiredness over the course of a session can changemovement characteristics.

The presence of multiple other sources of variability makes the task ofprogress monitoring challenging.

SUMMARY OF THE INVENTION

In an embodiment of the present invention a system is provided in whicha set of selected movements by an individual are measured over a periodof repetitions and sessions and the individual's progress is alsomeasured. In an embodiment of the system of the present invention, twoapproaches for monitoring patient movement are introduced forquantifying patient performance. Both approaches consider the healthypopulation as their reference and consider the difference, or what isreferred to in this application as the distance, between the patient'sdata and the healthy population as the measure of performance. Distancemeasures are defined to capture the performance of one repetition of anexercise or multiple repetitions of the same exercise. To capturepatient progress across multiple exercises, a quality measure andoverall score are defined based on the distance measures and are used toquantify the overall performance for each session. In the examplesillustrating the present invention, the effectiveness of these measuresin detecting patients' progress is evaluated on rehabilitation datarecorded from patients recovering from knee or hip replacement surgery.The measures of performance are able to capture the trend of patientimprovement over the course of rehabilitation. The trend of improvementis not monotonic and differs between patients.

In an embodiment of the present invention there is provided a method foranalysing an individual's motion through a computer programmed toprocess information, comprising the steps of:

-   -   measuring control linear acceleration and angular velocity,        using sensors on at least one control person performing a set of        repetitions of an exercise;    -   measuring individual linear acceleration and angular velocity,        using sensors on said individual performing a set of repetitions        of an exercise;    -   inputting the measured control linear acceleration and angular        velocity and individual linear acceleration and angular velocity        into said computer to convert to joint angle positions,        velocities and accelerations data for said individual and for        said at least one control person;    -   segmenting said data such that each segment begins with the        start of an exercise repetition and ends when the exercise        repetition is finished;    -   extracting feature vectors from said data such that a control        feature vector is V′_(H)=V_(H)(top_(features)) and an        individual's feature vector is V′_(P)=V_(P)(top_(features)) and        said top features differentiate the individual from the at least        one control person;    -   calculating a mean of the at least one control person's feature        vector such that μ_(H)=mean(V′_(H));    -   calculating a diagonal matrix of standard deviations for the at        least one control person's feature vector, such that        Σ_(H)=diag(std(V′_(H)));    -   and calculating a distance between one repetition of the        exercise performed by said individual and the at least one        control person's performance using        δ_(i)=(V′_(Pi)−μ_(H))^(T)Σ_(H)(V′_(Pi)−μ_(H)).

In an embodiment of the present invention there is provided a method formeasuring quality of a repetition set of said exercise above, wherein

${Q_{j} = \frac{\left( {\Delta_{Pj} - \mu_{\delta \; {Hj}}} \right)}{\sigma_{\delta \; {Hj}}^{a}}},$

-   -   and j is said exercise, Δ_(Pj) is the individual's said distance        measure for the repetition set of said exercise, μ_(δHj) is the        mean of the at least one control person's distance measure        vector δ_(Hj), a is the index that penalizes Q based on the        exercise difficulty, and σ_(δHj) is the standard deviation of        the at least one control person's distance measure vector        δ_(Hj).

In an embodiment of the present invention, a is 2.

In an embodiment of the present invention there is provided a method formeasuring the score for a set of more than one of said exercises j abovewherein the score is

$S = {\sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}\frac{\mu_{d_{H_{\Omega}}}\;}{\sigma_{d_{H_{\Omega}}}^{2}}} \right)^{2}} - \sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}Q_{\Omega}} \right)^{2}}}$

where Γ is said set of exercises, Ω is an exercise in the Γ, n_(Ω) isthe number of repetitions for exercise Ω, and Q_(Ω) is said qualitymeasure of claim 2 calculated for exercise a

In an embodiment of the present invention there is provided a method foranalysing an individual's motion through a computer programmed toprocess information, comprising the steps of:

-   -   measuring linear acceleration and angular velocity, using        sensors on at least one control person performing a set of        repetitions of an exercise;    -   measuring linear acceleration and angular velocity, using        sensors on said individual performing a set of repetitions of an        exercise;    -   inputting the measured control linear acceleration and angular        velocity and individual linear acceleration and angular velocity        into said computer to convert to joint angle positions,        velocities and accelerations data for said individual and for        said at least one control person;    -   segmenting said data such that each segment begins with the        start of an exercise repetition and ends when the exercise        repetition is finished;    -   extracting feature vectors based on the exercise repetition from        the at least one control person and said individual, wherein        said features differentiate the individual from the at least one        control person;    -   determining the distance between at least one control person and        said individual for that exercise repetition using the feature        vectors,    -   calculating the median of the exercise repetition for the at        least one control person and calculating the median of the        repetition set for the individual;    -   calculating the quality of a repetition set of an exercise based        on the individual's distance measure for each repetition set;        and    -   calculating the score of the exercise set based on the quality        of each of said repetition sets.

In an embodiment of the present invention there is provided acomputer-implemented method for selecting features which distinguishdata collected from an individual from data collected from at least onecontrol person comprising the steps of:

-   -   measuring linear acceleration and angular velocity, using        sensors on said at least one control person performing a set of        repetitions of an exercise;    -   measuring linear acceleration and angular velocity, using        sensors on said individual performing a set of repetitions of an        exercise;    -   inputting the measured control linear acceleration and angular        velocity and individual linear acceleration and angular velocity        into said computer to convert to joint angle positions,        velocities and accelerations data for said individual and for        said at least one control person;    -   segmenting said data such that each segment begins with the        start of an exercise repetition and ends when the exercise        repetition is finished;        and applying a Lasso analysis of the data collected to obtain        said features.

BRIEF DESCRIPTION OF THE FIGURES

These and other aspects of the present invention will be apparent fromthe brief description of the drawings and the following detaileddescription in which:

FIG. 1 is a flowchart of the overall system of an embodiment of thepresent invention.

FIG. 2 is a graphical representation of the a) repetition timeseries, b)repetition set, and c) exercise set according to an embodiment of thepresent invention.

FIG. 3 shows the joint angles' position, velocity, and acceleration forone repetition time series.

FIG. 4 is a flowchart of a Feature-Based Approach of an embodiment ofthe present invention.

FIG. 5 is a flowchart of an HMM-Based Approach of an embodiment of thepresent invention.

FIG. 6 are charts plotting patient and healthy population data for themost informative features for six patients for a) kneeextension/flexion; b) knee hip extension/flexion and c) squats, in anembodiment of the present invention.

FIG. 7 shows the measures of performance for each repetition time seriesof a patient performing a knee extension/flexion exercise.

FIG. 8 shows the results of distance measure analysis using thefeature-based approach for three patients for the exercises of kneeextension, knee hip extension/flexion and squats, in an embodiment ofthe present invention.

FIG. 9 shows the results of overall score using a feature-based approachfor an exercise set, combining individual distance measures of kneeextension, knee-hip extension and squat for three patients a), b), c),in an embodiment of the present invention.

FIG. 10 shows the results of distance measure analysis using the HMMapproach for three patients for the exercises of knee extension, kneehip extension/flexion and squats, in an embodiment of the presentinvention.

FIG. 11 shows the results of overall score using the HMM approach for anexercise set, combining individual distance measures of knee extension,knee-hip extension and squat for three patients a), b), c), in anembodiment of the present invention.

FIG. 12 is the correlation index for a) feature-based approach and b)HMM-based approach.

DETAILED DESCRIPTION

An embodiment of the overall system is shown in FIG. 1, in which rawdata is collected from a patient through the use of sensors. The rawdata collected is then filtered and segmented to estimate the jointangles and isolate the desired movement(s) to be monitored, consistingof joint angle positions, velocities, and accelerations, and toeliminate background movement. This system not only considers a singlejoint in the analysis, but simultaneously all of the joints in the leg.

The joint angle data is then segmented to extract each specific movementthat is being analyzed. A segment begins with the start of an exerciserepetition and ends when the exercise repetition is finished.Descriptive features are calculated for the joint angle positions,velocities, and accelerations. The data collected is then compared todata previously collected from healthy participants performing the sameexercises, and the healthy population data is used as a reference.

The patient data collected is then compared to the healthy populationdata. The difference between the patent data and the healthy populationdata is referred to as the distance. Distance measures quantify theperformance quality of: a) a single repetition of an exercise; b) a setof repetitions of the same exercise; and c) a set of differentexercises. These measures are used to provide the patients and thephysiotherapists with feedback on the patient's performance duringrehabilitation.

Data can be collected through the use of robotic exoskeletons or markerbased motion capture systems or wearable motion capture systems based onIMU sensors. In the case of IMU sensors, any sensor having a gyroscopeand an accelerometer will collect the required data, for example aShimmer™ sensor. Although a wired sensor will function for the purposesof this present system, it is preferable to use a wireless sensor. Forexample, one or more sensors connected through WiFi such as Bluetooth. Awireless system is preferable as a wired sensor may tug on the body ofthe wearer causing motion of the body or motion of the sensor which mayimpact the results of the data collected.

The data collected by the sensors is input onto a computing device. Anycomputing device having a processor may be used, for example, acomputer, laptop, tablet, phone, or computer processor.

The sensor or sensors continuously measure all movement of the wearer.In an embodiment of the present invention, the sensors measure theangular velocity and linear acceleration, while the system estimates thejoint positions, velocities and linear accelerations, as shown inFIG. 1. In an embodiment, the data collected by the sensor or sensors isconverted by way of a filter, such as an extended Kalman filter (EKF). Alinear filter is not used, because the relationship between theCartesian velocities/accelerations and jointangles/velocities/accelerations is non-linear. The extended Kalmanfilter is used, which extends the linear Kalman filter to allownon-linear observation and state equations. This is an algorithm thatuses a series of measurements observed over time, containing noise(random variations) and other inaccuracies, and produces estimates ofunknown variables that tend to be more precise than those based on asingle measurement alone.

The filter estimates the joint angles, velocities and accelerations, orother data, in regular intervals. It is understood that the intervals atwhich the algorithm estimates can be set when the sensor settings areset, for example every 30 to 50 Hertz. It is further understood that theintervals may vary beyond 30 to 50 Hz depending on the patient, thepurpose of using the system, the joints being measured, etc.

The filtered data is then segmented in order to extract from the rawdata collected the specific movement that is being analyzed. Thesegmentation of the raw data separates the data associated with thepatient performing the actual movement that is being analyzed from othermovements, such as stretching, shaking out the joint between exercises,stopping, walking around, etc.

The segmented data is next compared to a healthy population data usingeither a feature based method or a probabilistic or HMM method. Thedifference, or distance, between the patient's segmented data and thehealthy population data is then recorded and measured against subsequentsegmented data. Through comparing the distance of a patient over aperiod of time, patient progress can be monitored. The period of timecan be during a single session, or over a period of days, months, oryears.

The data collected through the use of analysing patient progressaddresses the questions of how to assess one repetition of one exerciseperformed by a patient; how to assess multiple repetitions of oneexercise performed by a patient; and how to combine the evaluations fromdifferent exercises and obtain a score that denotes the overallperformance of a patient in a single session.

In an embodiment of the invention, the motion data of patientsrecovering from knee or hip replacement surgery is collected in the formof, for example, angular velocities and linear accelerations, from whichjoint angle positions, velocities and accelerations are computed usingthe EKF. The motion data from a healthy population performing the sameset of exercises is also collected. To demonstrate the invention, thefollowing exercises are analyzed: knee extension/flexion while seated,knee and hip extension/flexion while supine, and squat. Shimmer sensorsare mounted at the knee and ankle to provide angular velocity and linearacceleration data at 128 Hz.

Referring to FIG. 2, repetition timeseries and the repetition setprovide the performance measures δ and Δ. The overall score S assessesan exercise set. During a knee extension exercise, the subject performsa full knee extension/flexion. During a knee hip extension exercise, thesubject lies on the ground and performs a knee hip extension/flexionsimultaneously. During a squat, the subject bends his knees and hipwhile standing. The repetition time series is shown at FIG. 2( a), therepetition set is shown at FIG. 2( b), and the exercise set is shown atFIG. 2( c). FIG. 3 illustrates the joint angles' position, velocity, andacceleration for one repetition time series.

The data is segmented such that one single repetition of a certainexercise is a repetition time series data ω=[γ₍₁₎γ₍₂₎ . . . γ_((T))],where T is the duration of the repetition for that exercise, and γ is avector of joint kinematics γ=[q₁ q₂ . . . {dot over (q)}₁ {dot over(q)}₂ . . . {umlaut over (q)}₁ {umlaut over (q)}₂ . . . ] Multiplerepetitions of the same exercise performed in the same session are therepetition set for that exercise Ω=[ω₁ . . . ω_(n)] where n is thenumber of repetitions. The set of multiple exercises performed in thesame session are the exercise set of that session Γ=[Ω₁ . . . Ω_(m)]where m is the number of different exercises performed in the session.

Two methods are described that measure the variability caused byimprovement and progress throughout the rehabilitation by reference to ahealthy population. In the first method, a feature-based approach,descriptive measures such as minimum, mean, or maximum are extractedfrom the joint angle position, velocity and acceleration time seriesdata. The second method, an HMM-based approach, relies on the featuresextracted from a generative model for the joint angle time series.Measures δ and Δ for assessing one and multiple repetitions of oneexercise are introduced based on a comparison between the healthypopulation and the patient. The overall score S is calculated as afunction of these measures for multiple exercises in one session.

Approach 1—Feature Based Approach

In an embodiment of the feature-based approach, descriptive measures areextracted from the joint angle time series. Referring to FIG. 4, in thefirst step of the feature-based approach, dta is collected from thepatent using IMU sensors, and the joint angles are estimated. The datais then segmented such that each segment begins with the beginning of anexercise, and the segment ends when one repetition of the exercise iscompleted. Statistical figures are extracted from the joint angles' timeseries, and the most descriptive features are selected using LASSO or KW(as described below). The measures of performance for a repetition timeseries, a repetition set, and an exercise set are then calculated usinga weighted distance between the patient population and the healthypopulation in the feature space.

The mean, minimum, maximum, skew and range of motion of joint anglepositions, velocities and accelerations along with the duration ofexercise for each repetition time series are considered as the featurevector:

v = [mean_(q 1)min_(q 1)max_(q 1)skew_(q 1)rom_(q 1)mean_(q 2)  …  duration]${skew}_{q_{i}} = \frac{\frac{1}{T}{\sum\limits_{j = 1}^{T}\; \left( {q_{i_{j}} - \mu_{q_{i}}} \right)^{3}}}{\left( \sqrt{\frac{1}{T}{\sum\limits_{j = 1}^{T}\; \left( {q_{i_{j}} - \mu_{q_{i}}} \right)^{2}}} \right)^{3}}$${rom}_{q_{i}} = {\max\limits_{q_{i}}{- {\min\limits_{q_{i}}.}}}$

This definition of the feature vector allows modeling the time series ofthe data using statistical features. This method permits fastcomputation and can capture the attributes of the time series from oneexample. The feature vectors are extracted for each repetition of everyexercise performed over the course of rehabilitation. These features arealso extracted from the healthy population data for the same exercise.Features that most reflect the changes throughout the rehabilitation arechosen using the LASSO method: Tibshirani, R. (1996) Regressionshrinkage and selection via the lasso. J. Royal. Statist. Soc B., Vol.58, No. 1, pages 267-288. KW refers to the Kruskal Wallis (KW) one-wayanalysis of variance, a non-parametric approach that determines howlikely it is for samples of multiple datasets to be from the samedistribution: W H Kruskal and W A Wallis. Use of ranks in one-criterionvariance analysis. Journal of the American Statistical Association,47(260):583-621, 1952.

The healthy data is the reference and the distance to the mean of thisdataset is the measure of progress for each exercise. The results ofdifferent exercises are normalized by mean and variance of theircorresponding healthy dataset. The measures are then combined to obtainan overall score for each patient's performance in a given session.

Feature Based Approach—Feature Selection

A linear model is used for feature selection, i.e., for identifyingwhich features change during the course of treatment. For the healthypopulation data, a linear model is used to estimate the suitability offeatures to discriminate between the healthy population and patientdata.

For a set of inputs f₁, f₂, . . . , f_(n), an output y and the followinglinear model:

ŷ=w ₀ +w ₁ f ₁ +w ₂ f ₂ +w ₃ f ₃ + . . . +w _(n) f _(n)

LASSO adjusts the weights w₀, . . . , w_(n) such that Σ(ŷ−y)² isminimized and Σ_(t=0) ^(n) w_(i)<t where t≧0 is a tuning parameter underLASSO. The second criterion drives the weights of non-informative inputsto zero. When w_(i) becomes zero the input f_(i) does not contribute inminimizing Σ(ŷ−y)².

The inputs f₁, f₂, . . . , f_(n) are the features of the repetition timeseries data and the output y is the corresponding session number. Thesession numbers are normalized between 0 and 1, such that 0 correspondsto a patient's first session, and 1 corresponds to a patient's lastsession. The session numbers are a linear function of suitable featuresthat allows one to find the features that are changed most withpatients' progress through the sessions.

Healthy population data in this regression is also considered. Label yfor the healthy population is considered to be 100 times larger thanpatients' last session. Introducing this outlier forces the regressionto be in the direction of the healthy population data and helpsdetecting the features that not only change with the progress of thepatients but also separate the healthy population from patients.

The value of y for the healthy population directly affects the value ofweights but the chosen features are not changed as long as the y valueis sufficiently large. When the weight w_(i) becomes zero it isinterpreted that feature f_(i) is uninformative and therefore is removedfrom the feature vector. The top features chosen by the LASSO techniqueare used for the subsequent analysis.

Feature Based Approach—Measure of Performance for Repetition Timeseries

To obtain a measurement for the performance of one exercise, the featurevectors are extracted for the patient (V_(P)) and healthy population(V_(H)) data as explained above.

V′ _(H) =V _(H)(top_(features))

V′ _(P) =V _(P)(top_(features))

Among the features chosen by LASSO, the ones with higher variance in thehealthy population are more informative. Therefore more weight is givento the more variant features in defining the distance measure. Toevaluate each repetition the distance between the patient repetition andthe healthy population data is found using the following equations:

μ_(H)=mean(V′H)

Σ_(H)=diag(std(V′ _(H)))

δ_(i)=(V′ _(Pi)−μ_(H))^(T)Σ_(H)(V′ _(Pi)−μ_(H))  Equation (1)

Where V′_(H) is the healthy population top feature vector, V′_(Pi) isthe patients' top feature vector for the ith repetition timeseries,μ_(H) is the mean of the healthy populations' top feature vector, Σ_(H)is the diagonal matrix of standard deviations for the healthy populationtop feature vector, δ_(i) is the distance between one repetition of theexercise performed and the healthy group's performance. As patientsimprove they get closer to the healthy data and therefore a decrease inthe value of δ over the course of rehabilitation indicates improvement.

Feature Based Approach—Measure of Performance for Multiple Repetitionsof the Same Exercise

The median of the distance measures (δ) calculated for one exercise overthe session is considered as the overall distance measure for therepetition set of that exercise:

Δ_(Ω)=median(δ_(Ω))

Where Δ_(Ω) is the overall performance of one exercise in one session.and δ_(Ω) is the vector of distance measures calculated for everyrepetition timeseries data ω_(i) in the repetition set Ω.

Feature Based Approach—Measure of Performance for a Combination ofExercises

The distance Δ_(Ω) obtained in measuring the performance for multiplerepetitions of the same exercise only describes the patients'performance for one exercise (i.e. Ω_(j)) in each session. There aremultiple exercises performed in each physiotherapy session (i.e. Γ) thatneed to be considered together for overall patient progress assessment.Quality and quantity are the two factors that affect scoring anexercise. The variance of distance measures calculated for the healthypopulation is used as a measure of exercise difficulty. The compensationstrategies performed by the healthy population have larger varianceswhen the exercise is more difficult and the distance measures of thehealthy population have larger variance for more difficult exercises.

The distance measures (δ) are calculated for every repetition timeseriesof the healthy population according to Equation 1 and are considered asthe comparison reference.

The healthy population distance measure vector δ_(Hj) is the vector ofthe distance measure calculated for every repetition timeseries ofexercise Ω_(j) in the healthy population data. The patient distancemeasures (Δ_(Pj)) are calculated for the repetition set of everyexercise Ω_(j) in the exercise set F. The mean and standard deviation ofthe healthy population distance measure vector δ_(Hj) are considered asthe measure of exercise difficulty and are calculated using thefollowing equations

μδ_(Hj)=mean(δ_(Hj))

σδ_(Hj) =std(δ_(Hj))

where μδ_(Hj) is the mean of the healthy population distance measurevector δ_(Hj) and σδ_(Hj) is the standard deviation of the healthypopulation distance measure vector

The measure of quality for a repetition set of an exercise j performedby the patient is:

$\begin{matrix}{{Q_{j} = \frac{\left( {\Delta_{Pj} - \mu_{\delta \; {Hj}}} \right)}{\sigma_{\delta \; {Hj}}^{a}}},} & {{Equation}\mspace{14mu} (2)}\end{matrix}$

where Δ_(Pj) is the patient's distance measure for the repetition set ofexercise j, and a is the index that penalizes Q based on the exercisedifficulty, i.e. larger a increases the imporatance of exercisedifficulty. In an embodiment of the present invention, a is 2. A perfectperformance over any repetition set Ω results in a value of zero for theoverall distance measure (Δ_(Ω)=0). The overall score for the patient ina specific session is calculated as the difference between the norm ofthe score resulting from a perfect performance and the norm of theweighted quality measures. The quality measure Q_(j) of an exercise j isweighted by its number of repetitions. The score of the patient for agiven session is calculated using the following equation

$\begin{matrix}{S = {\sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}\frac{\mu_{d_{H_{\Omega}}}\;}{\sigma_{d_{H_{\Omega}}}^{2}}} \right)^{2}} - \sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}Q_{\Omega}} \right)^{2}}}} & {{Equation}\mspace{14mu} (3)}\end{matrix}$

where Γ is the exercise set, Ω is an exercise in the Γ, n_(Ω) is thenumber of repetitions for exercise Ω, and Q_(Ω) is the quality measurecalculated for exercise Ω using Equation 2. The score S is formulatedsuch that performing a difficult exercise in a session would improve apatient's score. Furthermore, exercises with more repetitions in onesession are the main focus of that session and therefore, the qualitymeasures Q are weighted by the number of repetitions for each exercise.The score S is defined as the difference between a perfect weightedquality measure and the patient's weighted quality measure henceprogress results in smaller values for this measure.

The healthy population's distance values are often small and have asmall variance compared to patient data. In some cases the healthypopulation's distance measure variance σδ_(Hj) becomes smaller than 1.In Equation 2, we want to normalize the quality measure according to thehealthy population performance. To avoid dividing the quality measurewith a value less than 1, all the δ values are scaled uniformly for thehealthy population and patient data such that all variance values of thehealthy population's distance measures become greater than 1. Thealgorithm flexibility in defining any exercise set allows us tocalculate the overall score for any arbitrary set of exercises.

Approach 2—HMM Approach

In the second method, the Hidden Markov Model (HMM) based approachrelies on features extracted from HMMs modeling the joint angle timeseries. Referring to FIG. 5, after motion recording and segmentation, athree-state HMM is trained for each repetition set. The mean andstandard deviation of the observation of each state are considered asfeatures. The most informative features are selected using LASSO or KW,and the most informative features are selected using KW, and themeasures of performance for the repetition set, and the overall exerciseset are calculated.

HMMs are trained on the repetition set of each exercise for the healthyand unhealthy patient populations. The HMM is described in L. R.Rabiner, “A tutorial on hidden markov models and selected applicationsin speech recognition”, Proceedings of the IEEE, vol. 77, no. 2, pp.257-286, 1989.

The HMMs are learned for each member of the healthy populationseparately. For the data of each patient, individual HMMs are learnedfor every repetition set of each session (some sessions include morethan one repetition set of an exercise). The observations of the HMMsare the position, velocity, and acceleration of the joint angles, andthe mean and variance of the observation distributions in each state areconsidered as the feature vector.

LASSO feature selection is used to choose the most informative features,similar to the approach described in Feature Selection of the FeatureBased Approach 1. The distance measure is calculated using Equation 1.The quality measure Q is calculated using Equation 2 and the overallperformance score S is calculated using Equation 3.

HMM Approach—Feature Selection

HMMs are trained for the repetition sets of each exercise for thehealthy population and the patient data. The observations are the jointangle positions, velocities, and accelerations. Each repetition set ismodelled using a 3 state, left to right model. States 1, 2, and 3correspond to the stages: attempt to reach the desired posture, reachingthe desired posture and pausing, and returning to the starting posture.

The feature vector includes the parameters of the observationdistributions, which are the means and variances of joint anglepositions, velocities and accelerations for the three states

$V = \begin{bmatrix}\mu_{{state}_{1_{q\; 1}}} & \sigma_{{state}_{1_{q\; 1}}} & \mu_{{state}_{2_{q\; 1}}} & \ldots & \mu_{{state}_{3_{{\overset{\_}{q}}_{5}}}} & \sigma_{{state}_{3_{\overset{\_}{q}5}}}\end{bmatrix}$

The feature selection is performed using LASSO as described in theFeature Based Approach.

HMM Approach—Measure of Performance for a Repetition Timeseries

An advantage of the HMM is that it can be trained using severaltimeseries and that it represents the most likely timeseries. For thisreason, training HMMs for each repetition timeseries is not necessary toassess the performance for an exercise set or to assess an overallscore.

HMM Approach—Measure of Performance for Multiple Repetitions of the SameExercise

Of the top features chosen by LASSO, those that are highly variant inthe healthy population indicate the rehabilitation process better.Therefore, distance measures are calculated as in the Feature BasedApproach. The ten features chosen by LASSO are chosen as the topfeatures. The distance measure Δ_(HMM) for a repetition set is obtainedusing the HMM-based feature vectors and Equation 1.

HMM Approach—Measure of Performance for a Combination of Exercises

Based on Δ_(HMM) the procedure follows the Feature Based Approach. Thequality measure is defined by Equation 2 and is based on the patient'sdistance measure Δ_(HMM) for a repetition set. The overall score S_(HMM)of a session is obtained using Equation 3.

In this example, motion data is collected using Shimmer sensors mountedat the knee and ankle providing angular velocity and linear accelerationdata (128 HZ). Position q, velocity q and acceleration {umlaut over (q)}of five joint angles consisting of knee flexion, knee rotation, hipflexion, hip abduction, and hip rotation are estimated from the sensordata based on a kinematic model and an Extended Kalman Filter describedat J. F. Lin and D. Kulic, “Human pose recovery using wireless inertialmeasurement units”, Physiological measurement, vol. 33, no. 12, p. 2099(2012). The joint angle variables are denoted as

γ[q ₁ {dot over (q)} ₁ {umlaut over (q)} ₁ , . . . q ₅ {dot over (q)} ₅{umlaut over (q)} ₅]

Results Using Feature Based Approach 1

The LASSO technique described above is used for feature selection. FIG.6 shows that patient data and healthy population data are clearlyseparable for the most informative features. The star indicates thefirst day the patients have performed the exercises and the triangleindicates the last day the patients have performed the exercises (only 1session available for patient 5). q1 is the joint angle corresponding tohip extension and q4 is the joint angle corresponding to knee extension.FIG. 6 shows the distribution of the repetition timeseries of thehealthy population and the training subset of the patient data over thetwo features selected by LASSO that have the largest variance in thehealthy population. The clusters of the healthy population data and thepatient data are clearly separable. Furthermore, FIG. 6 shows that apatient's progress is in the direction of the variance of the healthypopulation data and moves towards the mean of the healthy population asthe patients improve their performance during rehabilitation. Whenpatient data is not available for training, features may be selected bylargest variance within the healthy population data.

FIG. 6 shows the values of δ for the second session of patient 2,illustrating that the feature-based approach captures the variation inexercise performance over the course of multiple repetitions.

The distance measure Δ provides information about the progress for eachrepetition set Ω through the rehabilitation. When patients improve, thedistance measure decreases towards zero. FIG. 8 shows the results forthe distance measure δ calculated using the feature-based approach,shown for three exercises. The circle illustrates the median of thedistance measures (i.e. Δ) in each session and the bar depicts thevariance of the distance measures δ in one session. The size of thecircle indicates the number of repetitions available in each repetitionset. FIG. 8 shows the calculated distance measure Δ and the distributionof 6 for the 3 example patients. The exercise regimen is specific toeach patient. The exercises are performed in a subset of the sessions,e.g., patient 2 performs knee extensions in session 1, 2, 7, 8 and 10.Furthermore factors such as pain, fatigue, psychological status, andenvironmental conditions contribute to patients' performance and itcannot be expected that the patient progress increases monotonically.

For all three patients an overall improvement over the course of thephysiotherapy treatment can be observed. Some patients show rapidprogress and are discharged early, e.g., patient 18 (in FIG. 8 d). Thedistance measure for a repetition set is more reliable when the numberof repetitions available for that exercise is larger. The feature-basedapproach generalizes to unseen patient data, e.g., the data of patients8 and 18 was not used for the feature selection and the distance measureΔ shows the patient's progress throughout the rehabilitation asdemonstrated in FIG. 8 g. The quality measure Q and the overall score Sfor each session are obtained according to Equations 2 and 3 using theoverall distance measure Δ calculated for every repetition set of eachsession. As patients improve, the overall score increases from negativevalues towards zero.

FIG. 9 shows an overall score S calculated for an exercise set,combining individual distance measures Δ of knee extension, knee-hipextension, and squat. The size of the marker indicates the number ofexercises available in each session. The top line shows the best scoreof the patients in their last physiotherapy session.

FIG. 9 shows the score measures for each patient. It can be seen fromthe figures that the trend of the score shows progress but there aresome inconsistencies in patient 2, session 8 (in FIG. 9 a). Theseinconsistencies are caused by a small number of performed exerciserepetitions.

Due to differences in health status, the exercise regimen of eachsession is different from one patient to the other. Among the threeexercises chosen for analysis, there are sessions where only one ofthese exercises is performed and therefore the score is based solely onthe performance quality of that single exercise. This results ininconsistencies in the improvement trend of the score measure since apoor performance in one exercise is not an accurate measure of thepatient's overall status. The score measure estimates the patient'soverall status more accurately when more exercise data from each sessionis available.

For patient 18, who had a fast recovery and was discharged after only 3sessions, the score is able to capture the rapid trend of progress inFIG. 9 b. As can be seen in FIG. 6, the top features capturing patientprogress are highly variant in the healthy population. If patient datais unavailable, the feature-based approach can be performed using onlythe most variant features in the healthy population. Since featureselection is not performed on patient population data, variabilities dueto initial posture may be selected as highly variant features among thehealthy population. To avoid this, features obtained from joint anglepositions were removed from the considered feature vector. The featurevector for each repetition timeseries data becomes

V=[μ _(q1) min_(q1) max_(q1) skew_(q1) rom_(q1) μ_(q2) . . . duration]

Among the first fifteen most variant features in the healthy population,those that correlate least with each other (less than 0.5) are chosen asthe top features. The distance measure is calculated for each repetitiontimeseries using Equation 1. The median of the distance measures δcalculated for the repetition timeseries is considered as the overalldistance measure Δ for a repetition set.

The quality measures and the overall score of each session arecalculated using Equations 2 and 3. FIG. 12 a shows the correlationindex between approaches with feature selection on healthy and unhealthydata and approaches with feature selection only on healthy data are formost patients over 65.

FIG. 12 a shows the correlation between the overall score from thefeature-based approach when using healthy and patient population datafor feature selection and when using only healthy population data forfeature selection.

As can be seen in FIG. 12 a, the results correlate highly (over 0.65)for most patients. Even though feature selection based only on healthypopulation data does not take compensation strategies specific to thepatient population into account, the extension using only healthypopulation data for feature selection can detect patient progress. Whenthe overall performance of a patient is constantly high (patient 7 and15) or low (patient 5) over the course of the rehabilitation, changes inthe scores are small. In these cases, the correlation index can be low,because the two techniques differ when assessing small changes inperformance.

Results Using HMM-Based Approach

HMMs are trained for the down sampled repetition sets for every healthysubject and every session of each patient. The feature vectors areobtained from the states of the HMMs and most informative features areselected. The distance measure Δ is calculated for each repetition setusing Equation 1. As patients improve the distance measure decreasestowards zero. FIG. 10 shows the HMM-based distance measure Δ_(HMM) whichshows the trend in progress over the sessions, here illustrated forpatients 2, 8, and 18. The marker size indicates the number ofrepetitions available in the repetition set. FIG. 10 shows the resultsof calculations for the overall distance measure Δ. The features chosenby LASSO capture the progress in unseen data, e.g., shown for patient 8in FIG. 10 g.

The quality measure Q of each exercise in one session is calculatedusing Equation 2 and the overall score S for the exercise set performedin each session is calculated using Equation 3. The overall scoreincreases from negative values towards zero as the patients improve.FIG. 11 shows the overall score S_(HMM) which shows the trend ofprogress during rehabilitation. The marker size indicates the number ofexercises available for each session. The top line shows the best scoreof the patients in their last session of performing the three exercises.FIG. 11 shows the scores S_(HMM) for each patient. The scores show anoverall trend of improvement for most patients. As before, thereliability of the score measure depends on the number of availableexercises, i.e., outliers are usually observed when only one exercise isavailable to calculate the score. The features chosen by LASSOgeneralize well to the unseen data e.g., patient 8 whose trend ofimprovement is captured by the approach (in FIG. 11 c).

The approach captures the rapid improvement of patient 18 (in FIG. 11b), who is discharged earlier than other patients. Healthy populationdata is sufficient for feature selection for the HMM-based approach.

Among the first fifteen most variant features in the healthy population,those that correlate least with each other (less than 0.5) are chosen asthe top features. The distance measure Δ_(HMM) for each repetition setis calculated using Equation 1. The quality measure Q_(HMM) for eachrepetition set is calculated using Equation 2 and the overall scoreS_(HMM) is calculated for each patient and each session using Equation3.

The correlation between the overall score using healthy and patientpopulation data for feature selection and using only healthy populationdata for feature selection is above 0.65 for most patients (see FIG. 12b). Negative correlation indices are observed when the changes in apatient's progress are small.

For both the feature-based and the HMM-based approach, the distancemeasure Δ for a repetition set and the overall score S for an exerciseset assess patient progress. The feature-based approach is faster tocompute whereas the HMM-based approach provides details about each stageof a motion.

Since the exercise regimen differs between patients and largevariability is observed in the patient population, it is difficult touse only patient data for learning a model which estimates patientprogress and generalizes to new patients. The proposed approachesachieve generalization to new patients by including healthy populationdata as reference. Feature selection is based on patient and healthypopulation data and an extension is evaluated using only healthypopulation data for feature selection. This extension is beneficial whenincluding new exercises into the exercise regimen without having accessto patient data.

Furthermore the score measure formulations can be applied to any set ofexercises as long as the corresponding healthy population data isavailable. This flexibility enables the physiotherapist to includepatient specific or novel exercises requiring only a healthy referenceset. The score measure is formulated in a way to handle individualexercise regimens and a variable number and type of exercises.

Quantified and continuous measure of performance can be beneficial formonitoring patient progress during the course of physiotherapyrehabilitation. This work introduces two approaches, feature-based andHMM-based, for capturing the continuous change in patient data. Adistance measure is introduced as a measure of performance for arepetition timeseries and repetition set. The overall score is thencalculated for the exercise set in each session and captures the overallperformance of the patient. The proposed approach can be trained basedon a subset of patient data and healthy population data, or usinghealthy population data alone, unlike most classifier methods. The twoapproaches of the present invention are evaluated on data of exercisescommonly performed after hip or knee replacement surgery. The resultsshow that the two approaches are able to track patient progress over thecourse of treatment.

This invention has applications other than rehabilitation, for example,it is used to measure progress (improvement in performance) duringsports training, or during motion analysis/intervention/training forimproving ergonomics in the workplace (avoiding workplace injuries).

The present invention when utilized for individuals measuringimprovement in performance (e.g. for an athlete) or correction inperformance (e.g. for an assembly line worker to avoid strain orinjury), uses a comparison from a control person or control population.

The present invention utilizes software and computer processing tocollect, carry out the analysis and provide results. The distance andscore results can be displayed in a manner which aids in comprehensionfor the individual or for a therapist such as graphical visualization.

While embodiments of the invention have been described in the detaileddescription, the scope of the claims should not be limited by thepreferred embodiments set forth in the examples, but should be given thebroadest interpretation consistent with the description as a whole.

What is claimed is:
 1. A method for analysing an individual's motionthrough a computer programmed to process information, comprising thesteps of: measuring control linear acceleration and angular velocity,using sensors on at least one control person performing a set ofrepetitions of an exercise; measuring individual linear acceleration andangular velocity, using sensors on said individual performing a set ofrepetitions of an exercise; inputting the measured control linearacceleration and angular velocity and individual linear acceleration andangular velocity into said computer to convert to joint angle positions,velocities and accelerations data for said individual and for said atleast one control person; segmenting said data such that each segmentbegins with the start of an exercise repetition and ends when theexercise repetition is finished; extracting feature vectors from saiddata such that a control feature vector is V′_(H)=V_(H)(top_(features))and an individual's feature vector is V′_(P)=V_(P)(top_(features)) andsaid top features differentiate the individual from the at least onecontrol person; calculating a mean of the at least one control person'sfeature vector such that μ_(H)=mean(V′_(H)); calculating a diagonalmatrix of standard deviations for the at least one control person'sfeature vector, such that Σ_(H)=diag(std(V′_(H))); and calculating adistance between one repetition of the exercise performed by saidindividual and the at least one control person's performance usingδ_(i)=(V′_(Pi)−μ_(H))^(T)Σ_(H)(V′_(Pi)−μ_(H)).
 2. A method for measuringquality of a repetition set of said exercise in claim 1, wherein${Q_{j} = \frac{\left( {\Delta_{Pj} - \mu_{\delta \; {Hj}}} \right)}{\sigma_{\delta \; {Hj}}^{a}}},$and j is said exercise, Δ_(Pj) is the individual's said distance measurefor the repetition set of said exercise, μ_(δHj) is the mean of the atleast one control person's distance measure vector δ_(Hj), a is an indexthat penalizes Q based on exercise difficulty, and σ_(δHj) is thestandard deviation of the at least one control person's distance measurevector δ_(Hj).
 3. The method of claim 2, where a is
 2. 4. A method formeasuring the score for said individual in claim 2 for a set of morethan one of said exercises j wherein the score is$S = {\sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}\frac{\mu_{d_{H_{\Omega}}}\;}{\sigma_{d_{H_{\Omega}}}^{2}}} \right)^{2}} - \sqrt{\sum\limits_{\Omega \in \Gamma}^{\;}\; \left( {\frac{n_{\Omega}}{\sum\limits_{\Omega \in \Gamma}^{\;}n_{\Omega}}Q_{\Omega}} \right)^{2}}}$where Γ is said set of exercises, Ω is an exercise in the Γ, n_(Ω) isthe number of repetitions for exercise Ω, and Q_(Ω) is said qualitymeasure of claim 2 calculated for exercise Ω.
 5. A method for analysingan individual's motion through a computer programmed to processinformation, comprising the steps of: measuring linear acceleration andangular velocity, using sensors on at least one control personperforming a set of repetitions of an exercise; measuring linearacceleration and angular velocity, using sensors on said individualperforming a set of repetitions of an exercise; inputting the measuredcontrol linear acceleration and angular velocity and individual linearacceleration and angular velocity into said computer to convert to jointangle positions, velocities and accelerations data for said individualand for said at least one control person; segmenting said data such thateach segment begins with the start of an exercise repetition and endswhen the exercise repetition is finished; extracting feature vectorsbased on the exercise repetition from the at least one control personand said individual, wherein said features differentiate the individualfrom the at least one control person; determining the distance betweenat least one control person and said individual for that exerciserepetition using the feature vectors, calculating the median of theexercise repetition for the at least one control person and calculatingthe median of the repetition set for the individual; calculating thequality of a repetition set of an exercise based on the individual'sdistance measure for each repetition set; and calculating the score ofthe exercise set based on the quality of each of said repetition sets.6. A computer-implemented method for selecting features whichdistinguish data collected from an individual from data collected fromat least one control person comprising the steps of: measuring linearacceleration and angular velocity, using sensors on said at least onecontrol person performing a set of repetitions of an exercise; measuringlinear acceleration and angular velocity, using sensors on saidindividual performing a set of repetitions of an exercise; inputting themeasured control linear acceleration and angular velocity and individuallinear acceleration and angular velocity into said computer to convertto joint angle positions, velocities and accelerations data for saidindividual and for said at least one control person; segmenting saiddata such that each segment begins with the start of an exerciserepetition and ends when the exercise repetition is finished; andapplying a Lasso analysis of the data collected to obtain said features.